1. The Field of the Invention
The present invention relates generally to a method and system for annotating an image, and more particularly, but not necessarily entirely, to a computerized method and system for creating, storing, displaying and associating structured, vector based, interactive visual annotations and interactive visual notes (also referred as “IVN”) with a digital image.
2. Description of Background Art
Annotation of visual material is a necessary activity, one that represents an important part of teaching, publishing and communicating visual information. Since the widespread adoption of computers and especially the Internet, the use of digital images in computer-assisted presentations has grown tremendously. Doctors, educators, geologists, architects, engineers, scientists are examples of professions where the use of digital images is becoming more widespread.
Image annotating, in a broad sense, includes any technique which allows an author to label, point to or otherwise indicate some feature of the image that is the focus of attention, including textual commentary. Providing an individual with the ability to add symbols, labels and captions to describe the contents of an image or to convey a concept and direct the viewer to important features of an image has been established for decades. It has been long accepted that assigning captions or a definition and providing an option to write a legend that further describes a region of interest that is unique to an image allows an author to convey intellectual information regarding the structures in the image itself. Traditional methods for annotating images have allowed authors to place pointers, textual information and labels to indicate structures contained in an image but that information remains static. Moreover, to change annotated features on an image often requires that the author scrape off the rub-on labels and start over or to reprint the image and start over with annotating the features of the image.
Today, digital image data is generated from a vast array of electronic devices and with the wide acceptance of the use of computers to accomplish the tasks of annotation gives rise to that fact that many applications have been designed to give authors the ability to annotate electronic image data. The same traditional sets of tools that have allowed authors to prepare image data for publication have essentially been reproduced in an electronic environment and can be used in addition to, or completely replace, traditional tools for annotating images.
Digital images are typically stored as raster images, also referred to as bitmap images. Examples of formats that are raster based include JPEG, GIF, BMP, PNM, TIFF, PPM, PNG and many others. Raster images are generally defined to be a rectangular array of regularly sampled values, known as pixels. Each pixel (picture element) has one or more numbers associated with it, generally specifying a color which the pixel should be displayed in. Most formats for raster images, including those mentioned above, compress the pixel information to shrink the size of the data needed to encode the image.
Authors of digital material are finding that creating, presenting, and cataloging digital images is a difficult task despite the technologic improvements. Visually annotating or illustrating digital images with symbols and text is a fundamental task many users of images must perform when preparing material for illustration. For example, clinicians and biomedical investigators must make visual annotations when preparing material for illustration.
Annotating visual media has evolved from scratch-on LETRASET® dry transfer labeling to using expensive, sophisticated and complicated image manipulation computer software like ADOBE® PHOTOSHOP® or MACROMEDIA® FREEHAND® software. At the same time, the need to illustrate images with annotations requires very little (symbols, labels, shapes and arrows) and remains a simple task. While rub-on labels certainly have large drawbacks, i.e., they cannot be used for annotating digital images, they embody the simplicity of annotating an image quickly with the necessary information. Sophisticated software, while capable of generating simple annotations, requires a high degree of skill and knowledge to navigate the complexity of options and functions to achieve what is, in the end, a simple task. Moreover, the previously available systems and methods do not promote interactivity with a user, neither in their output nor in their presentation. Thus, simplicity, interactivity and low cost continue as unsatisfied objectives for the process of effectively annotating visual digital material despite technologic improvements.
Not only is most image manipulation software functional overkill for creating simple annotations, this software flattens images where the annotations are “fixed” to the image much like rub-on labels. The flattening of image annotations causes several problems that also existed with rub-on labels: editing difficulties, poor image quality, lack of interactivity and information loss. These problems are further illustrated below.
Annotations are not easily edited in a flattened image. The process of editing a flattened image requires using the original image—often in a proprietary format—in the native environment of the authoring software. This process requires locating the original (not the presentation image currently in use) image or images—typically on a local hard drive—making the changes and then redistributing that image to the various publishing targets: Internet/WWW, paper-based copies, and so on. If the original image is lost then the annotations must be completed again from scratch. Those that have used this process—locating an image, making changes, then redistributing the image—can attest to the time and frustration involved.
In the previously available systems and methods, annotations when flattened become part of the raster-based (drawn with pixels) image as opposed to being stored as vector (drawn in true physical space) information. As the raster annotations are re-scaled (zoom in or out) their appearance often become incomprehensible.
Flattening of annotations to an image means not only that the annotations cannot be scaled accordingly, it means that the annotations cannot be manipulated in other ways, such as, creating interactive presentations for the purpose of communicating a visual relationship or integrating the annotations into a learning assessment tool. Since the Internet has emerged as a viable medium to deliver educational materials, presentors are more often using the Internet to provide computer-assisted presentations of educational material. Moreover, providing computer-assisted presentations has become easier than ever with the advancements in technology, computer hardware, software and improvements in the Internet and World Wide Web as delivery a mechanism. For example, in an illustration of brain anatomy it may be necessary to illustrate the neurology and gross anatomy side-by-side. But it may also be useful to hide or turn off the gross anatomy in order to illustrate the neurology then turn the gross anatomy back on to illustrate the relationship(s) between the two groupings. This scenario could be solved with raster images, however, it would require three images—one with neurology, one with gross anatomy, and one with both. Thus, there is four times the effort to produce this basic level of interactivity. Additionally, If these images are being viewed on the Internet it would mean three times longer wait in addition to the labor and disk space utilized in producing three images. As the interactivity of an educational presentation increases the effort involved with raster images will grow exponentially.
The fourth and possibly the most significant problem arising from flattened annotations is the loss of information. For example, in the situation of medical research and instruction, scientists, teachers, physicians, residents and students go to a network, such as the Internet, expecting to find resources on a particular topic by entering a keyword or phrase representing the subject or title of their objective. In order for a resource to be found, information about that resource must be indexed or cataloged like the age-old library card catalog.
Annotated images are one example of the valuable resources that need to be integrated into a catalog or index in order to be found and to realize their value. Annotated images offer more value than the base image in that there is intellectual or authored content assigned to the features of the image providing instructive value beyond the image itself. The annotations are part of the content. In order to index the annotated image this and other information—metadata—about the image (subject, keyword, format, date created, copyright, etc.) must be cataloged. However, annotations that are flattened to the image are not available for cataloging. Either the content of the annotations is lost or the annotations must be entered again into another system and associated with the image. This de-coupling of content from the image and re-entry of the annotations into a separate storage system which is required when using the previously available systems and methods results in a more labor intensive, complex and disjoint procedure.
Another disadvantage to using a flattened image is the inability to allow multispecialty authoring. Multispecialty authoring is the ability to add visual annotations, stored as groups, according to authorship. Often it is necessary that several different individuals annotate the same image. For example, several different specialties in the medical field may need to annotate an x-ray image. Using a flattened image, this would be extremely difficult.
Another drawback to flattened images is that it is difficult to modify annotated images to make them context appropriate. Annotated images often contain annotations that are not appropriate for the persons viewing the image for a variety of reasons. For example, this might include information that is prohibited from being disseminated by privacy laws or simply information that is irrelevant given the audience. Removing or hiding from view the annotations from a flattened image is not efficient due to the fact that the annotations are embedded in the image.
Still another drawback to the use of flattened images is the difficulty in reusing the annotated image. Reusing images in a variety of different mediums is an attractive option for authors. Authors will often decide to publish annotated image data to a variety of media. Some will publish in traditional peer reviewed journals and textbooks and others will want to publish annotated material to the World Wide Web. Moreover, the context in which an image will appear may require that the content, both image and annotations, be presented differently. When working from a flattened image, a great deal work must be duplicated to provide suitable flattened images for each context. Thus, it is in the best interest of the system architect and the author to create an archive image with associated annotations and store annotations as vector information or text data.
Reuse (linking or referencing) enables authors to easily and accurately link information, and then maintain links across document revisions and system changes. Adhering to a reuse policy could potentially reduce storage costs, and reuse rather than duplication promotes sharing of existing authored material rather than recreating it. The traditional known forms of output-based reuse include print publication, color plates, 35 mm slides, and the many forms of digital publication (e.g., PDF, HTML, etc.). Another form of reuse is in-system digital reuse of existing information. For example, a user might add multiple sets of annotations to an image and then desire to activate or inactivate the various sets of annotations to customize the image for use in different contexts, such as on a world wide web page, in a print document, or in the portable document format (PDF).
As discussed above, the previously available methods and systems are thus characterized by several disadvantages that are addressed by the present invention. The present invention minimizes, and in some aspects eliminates, the above-mentioned failures, and other problems, by utilizing the methods and structural features described herein.